Forensic face photo-sketch recognition using a deep learning-based architecture

Numerous methods that automatically identify subjects depicted in sketches as described by eyewitnesses have been implemented, but their performance often degrades when using real-world forensic sketches and extended galleries that mimic law enforcement mug-shot galleries. Moreover, little work has been done to apply deep learning for face photo-sketch recognition despite its success in numerous application domains including traditional face recognition. This is primarily due to the limited number of sketch images available, which are insufficient to robustly train large networks. This letter aims to tackle these issues with the following contributions: 1) a state-of-the-art model pre-trained for face photo recognition is tuned for face photo-sketch recognition by applying transfer learning, 2) a three-dimensional morphable model is used to synthesise new images and artificially expand the training data, allowing the network to prevent over-fitting and learn better features, 3) multiple synthetic sketches are also used in the testing stage to improve performance, and 4) fusion of the proposed method with a state-of-the-art algorithm is shown to further boost performance. An extensive evaluation of several popular and state-of-the-art algorithms is also performed using publicly available datasets, thereby serving as a benchmark for future algorithms. Compared to a leading method, the proposed framework is shown to reduce the error rate by 80.7% for viewed sketches and lowers the mean retrieval rank by 32.5% for real-world forensic sketches.peer-reviewe

Exploiting color-depth image correlation to improve depth map compression

The multimedia signal processing community has recently identified the need to design depth map compression algorithms which preserve depth discontinuities in order to improve the rendering quality of virtual views for Free Viewpoint Video (FVV) services. This paper adopts contour detection with surround suppression on the color video to approximate the foreground edges present in the depth image. Displacement estimation and compensation is then used to improve this prediction and reduce the amount of side information required by the decoder. Simulation results indicate that the proposed method manages to accurately predict around 64% of the blocks. Moreover, the proposed scheme achieves a Peak Signal-to-Noise Ratio (PSNR) gain of around 4.9-6.6 dB relative to the JPEG standard and manages to outperform other state of the art depth map compression algorithms found in literature.peer-reviewe

Light field image processing : overview and research issues

Light field (LF) imaging first appeared in the computer graphics community with the goal of photorealistic 3D rendering [1]. Motivated by a variety of potential applications in various domains (e.g., computational photography, augmented reality, light field microscopy, medical imaging, 3D robotic, particle image velocimetry), imaging from real light fields has recently gained in popularity, both at the research and industrial level.peer-reviewe

Matching software-generated sketches to face photographs with a very deep CNN, morphed faces, and transfer learning

Sketches obtained from eyewitness descriptions of criminals have proven to be useful in apprehending criminals, particularly when there is a lack of evidence. Automated methods to identify subjects depicted in sketches have been proposed in the literature, but their performance is still unsatisfactory when using software-generated sketches and when tested using extensive galleries with a large amount of subjects. Despite the success of deep learning in several applications including face recognition, little work has been done in applying it for face photograph-sketch recognition. This is mainly a consequence of the need to ensure robust training of deep networks by using a large number of images, yet limited quantities are publicly available. Moreover, most algorithms have not been designed to operate on software-generated face composite sketches which are used by numerous law enforcement agencies worldwide. This paper aims to tackle these issues with the following contributions: 1) a very deep convolutional neural network is utilised to determine the identity of a subject in a composite sketch by comparing it to face photographs and is trained by applying transfer learning to a state-of-the-art model pretrained for face photograph recognition; 2) a 3-D morphable model is used to synthesise both photographs and sketches to augment the available training data, an approach that is shown to significantly aid performance; and 3) the UoM-SGFS database is extended to contain twice the number of subjects, now having 1200 sketches of 600 subjects. An extensive evaluation of popular and stateof-the-art algorithms is also performed due to the lack of such information in the literature, where it is demonstrated that the proposed approach comprehensively outperforms state-of-the-art methods on all publicly available composite sketch datasets.peer-reviewe

A support vector machine approach for detection and localization of transmission errors within standard H.263++ decoders

Wireless multimedia services are increasingly becoming popular boosting the need for better quality-of-experience (QoE) with minimal costs. The standard codecs employed by these systems remove spatio-temporal redundancies to minimize the bandwidth required. However, this increases the exposure of the system to transmission errors, thus presenting a significant degradation in perceptual quality of the reconstructed video sequences. A number of mechanisms were investigated in the past to make these codecs more robust against transmission errors. Nevertheless, these techniques achieved little success, forcing the transmission to be held at lower bit-error rates (BERs) to guarantee acceptable quality. This paper presents a novel solution to this problem based on the error detection capabilities of the transport protocols to identify potentially corrupted group-of-blocks (GOBs). The algorithm uses a support vector machine (SVM) at its core to localize visually impaired macroblocks (MBs) that require concealment within these GOBs. Hence, this method drastically reduces the region to be concealed compared to state-of-the-art error resilient strategies which assume a packet loss scenario. Testing on a standard H.263++ codec confirms that a significant gain in quality is achieved with error detection rates of 97.8% and peak signal-to-noise ratio (PSNR) gains of up to 5.33 dB. Moreover, most of the undetected errors provide minimal visual artifacts and are thus of little influence to the perceived quality of the reconstructed sequences.peer-reviewe

Accurate modelling of Ka-band videoconferencing systems based on the quality of experience

This work formed part of the project TWISTER, which was financially supported under the European Union 6th Framework Programme (FP6). The authors are solely responsible for the contents of the paper, which does not represent the opinion of the European Commission.Ka-band satellite multimedia communication networks play important roles because of their capability to provide the required bandwidth in remote places of the globe. However, because of design complexity, in practice they suffer from poor design and performance degradation because of being practically forced to guarantee acceptable end-user satisfaction in conditions of extremely low bit error rates, which is emphasised with the vulnerability of compressed video content to transmission errors, often impossible to be applied during the service development phase. A novel discrete event simulation model is presented, which provides performance estimation for such systems based on subjective measurement and a better quality of experience. The authors show that the proposed model reduces implementation cost and is flexible to be used for different network topologies around the globe.peer-reviewe

A hybrid error control and artifact detection mechanism for robust decoding of H.264/AVC video sequences

This letter presents a hybrid error control and artifact detection (HECAD) mechanism which can be used to enhance the error resilient capabilities of the standard H.264/advanced video coding (AVC) codec. The proposed solution first exploits the residual source redundancy to recover the most likelihood H.264/AVC bitstream. If error recovery is unsuccessful, the residual corrupted slices are then passed through a pixel-level artifact detection mechanism to detect the visually impaired macroblocks to be concealed. The proposed HECAD algorithm achieves overall peak signal-to-noise ratio gains between 0.4 dB and 4.5 dB relative to the standard with no additional bandwidth requirement. The cost of this solution translates in a marginal increase in the complexity of the decoder. In addition, this method can be applied in conjunction with other error resilient strategies and scales well with different encoding configurations.peer-reviewe

Robust decoder-based error control strategy for recovery of H.264/AVC video content

Real-time wireless conversational and broadcasting multimedia applications offer particular transmission challenges as reliable content delivery cannot be guaranteed. The undelivered and erroneous content causes significant degradation in quality of experience. The H.264/AVC standard includes several error resilient tools to mitigate this effect on video quality. However, the methods implemented by the standard are based on a packet-loss scenario, where corrupted slices are dropped and the lost information concealed. Partially damaged slices still contain valuable information that can be used to enhance the quality of the recovered video. This study presents a novel error recovery solution that relies on a joint source-channel decoder to recover only feasible slices. A major advantage of this decoder-based strategy is that it grants additional robustness while keeping the same transmission data rate. Simulation results show that the proposed approach manages to completely recover 30.79% of the corrupted slices. This provides frame-by-frame peak signal-to-noise ratio (PSNR) gains of up to 18.1%dB, a result which, to the knowledge of the authors, is superior to all other joint source-channel decoding methods found in literature. Furthermore, this error resilient strategy can be combined with other error resilient tools adopted by the standard to enhance their performance.peer-reviewe

A robust error detection mechanism for H.264/AVC coded video sequences based on support vector machines

Current trends in wireless communications provide fast and location-independent access to multimedia services. Due to its high compression efficiency, H.264/AVC is expected to become the dominant underlying technology in the delivery of future wireless video applications. The error resilient mechanisms adopted by this standard alleviate the problem of spatio-temporal propagation of visual artifacts caused by transmission errors by dropping and concealing all macroblocks (MBs) contained within corrupted segments, including uncorrupted MBs. Concealing these uncorrupted MBs generally causes a reduction in quality of the reconstructed video sequence.peer-reviewe